Real-Time Object Detection Using YOLO and Jetson CSI Camera¶

📦 Importing Required Libraries¶

In this section, we import the essential libraries needed for camera access and object detection.

  • Ultralytics YOLO: used to load and run the object detection model
  • OpenCV (cv2): used for image processing and displaying frames
  • JetCam (CSICamera): provides an interface to access the Jetson CSI camera
  • time: used for handling timing and delays during execution

These libraries will allow us to capture live video from the Jetson camera and prepare it for object detection using YOLO.

In [ ]:
from ultralytics import YOLO
import cv2
from jetcam.csi_camera import CSICamera
import time

🤖 Loading the YOLO Model¶

In this step, we load a pretrained YOLO model that will be used for object detection.

  • The model yolo26n.pt is a lightweight version, where n stands for Nano, making it suitable for real-time applications on embedded systems like Jetson.
  • You can also experiment with other YOLO models such as YOLOv8 or larger variants (e.g., s, m, l) to compare performance and accuracy.
  • The fuse() function optimizes the model by combining certain layers, which improves inference speed without affecting accuracy.

This prepares the model to perform fast and efficient object detection on incoming camera frames.

In [ ]:
model = YOLO("yolo26n.pt") # load a pretrained model

model.fuse()  # Fuses the model for faster inference

🎥 Initializing the Jetson CSI Camera¶

We initialize the Jetson CSI camera using JetCam.

  • Resolution: 640×480
  • Frame rate: 30 FPS
  • capture_device=0: default camera

This setup provides a good balance for real-time object detection.

In [ ]:
camera = CSICamera(width=640, height=480, capture_width=640, capture_height=480, capture_fps=30,capture_device=0)

🖼️ Capturing an Image¶

In this step, we capture a single frame from the camera and prepare it for display.

  • camera.read() and camera.value: capture the current image
  • ipywidgets.Image: creates an image widget
  • bgr8_to_jpeg: converts the image to JPEG format

This verifies that the camera is capturing frames correctly.

In [ ]:
camera.read()
image = camera.value

import ipywidgets
from IPython.display import display
from jetcam.utils import bgr8_to_jpeg

image_widget = ipywidgets.Image(format='jpeg')

image_widget.value = bgr8_to_jpeg(image)

🔄 Real-Time Object Detection¶

In this step, we enable continuous camera streaming and perform real-time object detection using YOLO.

  • camera.running = True: starts the camera stream
  • camera.observe(...): updates the frame whenever a new image is captured
  • YOLO is applied to each frame with a confidence threshold of 0.5
  • FPS (frames per second) is calculated to monitor performance
  • Detected objects and FPS are displayed on the output image

This creates a live object detection system running directly on the Jetson.

In [ ]:
last_time = time.perf_counter()

def update_image(change):
    global last_time
    
    image = change['new']
    if image is None:
        return

    current_time = time.perf_counter()
    fps = 1 / (current_time - last_time)
    last_time = current_time

    results = model(image, conf=0.5, verbose=False)
    output_image = results[0].plot()
    output_image = cv2.putText(output_image, f"fps:{int(fps)}", (5,30),
                               cv2.FONT_HERSHEY_DUPLEX, 1, (125,246,55), 1)

    image_widget.value = bgr8_to_jpeg(output_image)

camera.running = True
camera.observe(update_image, names='value')
display(image_widget)

⏹️ Stopping the Camera Stream¶

In this step, we stop the real-time updates from the camera.

  • camera.unobserve(...): removes the update function linked to the camera stream

This stops the continuous object detection loop and freezes the display.

In [ ]:
camera.unobserve(update_image, names='value')

▶️ Resuming the Camera Stream¶

In this step, we re-enable the camera updates.

  • camera.observe(...): reconnects the update function to the camera stream

This resumes the real-time object detection.

In [ ]:
camera.observe(update_image, names='value')
In [ ]:
camera.unobserve(update_image, names='value')
In [ ]: